Accuracy Of Metacognitive Judgments Research Articles

Metacognitive effects of instructional visuals: the role of cue use and judgment type

Prior research suggests most students do not glean valid cues from provided visuals, resulting in reduced metacomprehension accuracy. Across 4 experiments, we explored how the presence of instructional visuals affects students’ metacomprehension accuracy and cue-use for different types of metacognitive judgments. Undergraduates read texts on biology (Study 1a and b) or chemistry (Study 2 and 3) topics, made various judgments (test, explain, and draw) for each text, and completed comprehension tests. Students were randomly assigned to receive only texts (text-only condition) or texts with instructional visualizations (text-and-image condition). In Studies 1b, 2 and 3, students also reported the cues they used to make each judgment. Across the set of studies, instructional visualizations harmed relative metacomprehension accuracy. In Studies 1a and 2, this was especially the case when students were asked to judge how well they felt they could draw the processes described in the text. But in Study 3, this was especially the case when students were asked to judge how well they would do on a set of comprehension tests. In Studies 2 and 3, students who reported basing their judgments on representation-based cues demonstrated more accurate relative accuracy than students who reported using heuristic based cues. Further, across these studies, students reported using visual cues to make their draw judgments, but not their test or explain judgments. Taken together, these results indicate that instructional visualizations can hinder metacognitive judgment accuracy, particularly by influencing the types of cues students use to make judgments of their ability to draw key concepts.

Advances in Understanding and Fostering the Accuracy of Metacognitive Judgments

Advances in Understanding and Fostering the Accuracy of Metacognitive Judgments

Bullshit receptivity, problem solving, and metacognition: simply the BS, not better than all the rest

People are often inaccurate in their predictions of performance on a variety of cognitive tasks. We tested whether receptivity to bullshit – the tendency to perceive meaningless statements as profound – would relate to the accuracy of metacognitive judgments on several problem-solving tasks. Individuals who were highly receptive to bullshit were less accurate in their predictions of performance on creative problem-solving tasks, but not on verbal analogy or recall tasks. Further, individuals with high BS receptivity were less able to discriminate between solvable and unsolvable problems when making metacognitive judgments. These findings support the possibility that the tendency to perceive semantic connections where none exist, as indicated by high bullshit receptivity, may lead to inaccurate predictions of performance on tasks that require noticing and utilizing distant semantic connections.

Testing pays off twice: Potentials of practice tests and feedback regarding exam performance and judgment accuracy

Two studies investigated the testing effects on performance and on metacognitive judgment accuracy in authentic learning settings. Across two educational psychology courses, undergraduate students had the opportunity to voluntarily participate in four different practice tests during the term—without feedback in Study 1 (N = 201 students) or with individual corrective feedback in Study 2 (N = 111 students). Across studies in real classroom settings with and without feedback, regression analyses indicated that a higher number of taken practice tests were related to higher performance in the final course exam and to two scores of metacognitive judgment accuracy (absolute accuracy and sensitivity). However, students’ preparation and post-processing practice tests, their perceived usefulness of tests for monitoring one’s performance, and metacognitive specificity differed depending on whether students received feedback or not. Overall, the studies convey considerable evidence on how participation in practice tests is related not only to performance but also to monitoring accuracy in authentic learning settings.

Explaining distortions in metacognition with an attractor network model of decision uncertainty.

Metacognition is the ability to reflect on, and evaluate, our cognition and behaviour. Distortions in metacognition are common in mental health disorders, though the neural underpinnings of such dysfunction are unknown. One reason for this is that models of key components of metacognition, such as decision confidence, are generally specified at an algorithmic or process level. While such models can be used to relate brain function to psychopathology, they are difficult to map to a neurobiological mechanism. Here, we develop a biologically-plausible model of decision uncertainty in an attempt to bridge this gap. We first relate the model’s uncertainty in perceptual decisions to standard metrics of metacognition, namely mean confidence level (bias) and the accuracy of metacognitive judgments (sensitivity). We show that dissociable shifts in metacognition are associated with isolated disturbances at higher-order levels of a circuit associated with self-monitoring, akin to neuropsychological findings that highlight the detrimental effect of prefrontal brain lesions on metacognitive performance. Notably, we are able to account for empirical confidence judgements by fitting the parameters of our biophysical model to first-order performance data, specifically choice and response times. Lastly, in a reanalysis of existing data we show that self-reported mental health symptoms relate to disturbances in an uncertainty-monitoring component of the network. By bridging a gap between a biologically-plausible model of confidence formation and observed disturbances of metacognition in mental health disorders we provide a first step towards mapping theoretical constructs of metacognition onto dynamical models of decision uncertainty. In doing so, we provide a computational framework for modelling metacognitive performance in settings where access to explicit confidence reports is not possible.

Mindreading and metacognition patterns in patients with borderline personality disorder: experimental study

Current psychopathology attempts to understand personality disorders in relation to deficits in higher cognition such as mindreading and metacognition. Deficits in mindreading are usually related to limitations in or a complete lack of the capacity to understand and attribute mental states to others, while impairments in metacognition concern dysfunctional control and monitoring of one’s own processes. The present study investigated dysfunctional higher cognition in the population of patients with borderline personality disorder (BPD) by analyzing the accuracy of metacognitive judgments in a mindreading task [reading the mind in the eyes Test (RMET)] and a subsequent metacognitive task based on self-report scales: a confidence rating scale (CR) versus a post-decision wagering scale (PDW). It turned out that people from the BPD group scored lower in the RMET. However, both groups had the same levels of confidence on the PDW scale when giving incorrect answers in the RMET test. As initially hypothesized, individuals with BPD overestimated their confidence in incorrect answers, regardless of the type of metacognitive scales used. The present findings indicate that BPD individuals show dysfunctional patterns between instances of mindreading and metacognition.

The effect of metacognitive training on confidence and strategic reminder setting.

Individuals often choose between remembering information using their own memory ability versus using external resources to reduce cognitive demand (i.e. 'cognitive offloading'). For example, to remember a future appointment an individual could choose to set a smartphone reminder or depend on their unaided memory ability. Previous studies investigating strategic reminder setting found that participants set more reminders than would be optimal, and this bias towards reminder-setting was predicted by metacognitive underconfidence in unaided memory ability. Due to the link between underconfidence in memory ability and excessive reminder setting, the aim of the current study was to investigate whether metacognitive training is an effective intervention to a) improve metacognitive judgment accuracy, and b) reduce bias in strategic offloading behaviour. Participants either received metacognitive training which involved making performance predictions and receiving feedback on judgment accuracy, or were part of a control group. As predicted, metacognitive training increased judgment accuracy: participants in the control group were significantly underconfident in their memory ability, whereas the experimental group showed no significant metacognitive bias. However, contrary to predictions, both experimental and control groups were significantly biased toward reminder-setting, and did not differ significantly. Therefore, reducing metacognitive bias was not sufficient to eliminate the bias towards reminders. We suggest that the reminder bias likely results in part from erroneous metacognitive evaluations, but that other factors such as a preference to avoid cognitive effort may also be relevant. Finding interventions to mitigate these factors could improve the adaptive use of external resources.

Effects of induced levels of prior knowledge on monitoring accuracy and performance when learning from self-regulated problem solving.

Both accurate monitoring and adequate control are crucial for effective self-regulation when learning from problem-solving tasks. Prior research has shown that self-regulated learning is especially harmful for low prior knowledge students, given their difficulties with accurate monitoring and control decisions. Although many studies have indicated that prior knowledge facilitates monitoring accuracy, none of these studies experimentally manipulated students’ prior knowledge. The purpose of the current study was to investigate whether experimentally induced levels of prior knowledge affect the accuracy of metacognitive judgments (i.e., ease-of-learning judgments [EOLs], and retrospective confidence judgments [RCJs]) when learning from problem-solving tasks, and whether this in turn moderates the effect of learner control on performance. To manipulate prior knowledge, we compared conditions in which undergraduate students (N = 165) watched either a video about Mendel’s biography (low prior knowledge) or a video about Mendel’s laws of heredity (high prior knowledge). After that, we presented students with a set of problem-solving tasks and manipulated the type of learner control between conditions (i.e., full control vs. restricted control over the selection of tasks). Results indicated that students with high induced prior knowledge were more accurate in their EOLs and RCJs, and achieved higher performance than students with low prior knowledge. However, monitoring accuracy did not moderate the effects of different types of learner control on performance, even though students used their RCJs as a basis for the task selections. These results suggest that monitoring accuracy is a prerequisite for successful self-regulated problem solving, but it is not sufficient. (PsycInfo Database Record (c) 2020 APA, all rights reserved)

Individual differences in local and global metacognitive judgments

Extensive research has been conducted to understand how accurately students monitor their studying and performance via metacognitive judgments. Moreover, the bases of students’ metacognitive judgments are of interest. While previous results are quite consistent regarding the importance of performance for the accuracy of metacognitive judgments, results regarding motivational and personality variables are rather heterogeneous. This paper reports on two studies that simultaneously examined the predictive power of several performance, motivational, and personality variables on metacognitive judgments. The studies investigated a set of judgments (local and global postdictions in Study 1 and global pre- and postdictions in Study 2) and accuracy scores (bias, sensitivity, and specificity) in two different settings. Individual differences in judgments and judgment accuracy were studied via hierarchical regression analyses. Study 1 with N = 245 undergraduate students identified performance and domain-specific self-concept as relevant predictors for judgments after test taking. This was consistently found for local and global judgments. Study 2 with N = 138 undergraduate students hence focused on domain-specific self-concept and extended results to predictions. Study 2 replicated results for global postdictions but not predictions. Specifically, before task processing, students’ judgments relied mostly on domain-specific self-concept but not on test performance itself. The studies indicate that different judgments and measures of judgment accuracy are needed to obtain comprehensive insights into individual differences in metacognitive monitoring.

Acute Physical Exercise Can Influence the Accuracy of Metacognitive Judgments

Acute exercise generally benefits memory but little research has examined how exercise affects metacognition (knowledge of memory performance). We show that a single bout of exercise can influence metacognition in paired-associate learning. Participants completed 30-min of moderate-intensity exercise before or after studying a series of word pairs (cloud-ivory), and completed cued-recall (cloud-?; Experiments 1 & 2) and recognition memory tests (cloud-? spoon; ivory; drill; choir; Experiment 2). Participants made judgments of learning prior to cued-recall tests (JOLs; predicted likelihood of recalling the second word of each pair when shown the first) and feeling-of-knowing judgments prior to recognition tests (FOK; predicted likelihood of recognizing the second word from four alternatives). Compared to no-exercise control conditions, exercise before encoding enhanced cued-recall in Experiment 1 but not Experiment 2 and did not affect recognition. Exercise after encoding did not influence memory. In conditions where exercise did not benefit memory, it increased JOLs and FOK judgments relative to accuracy (Experiments 1 & 2) and impaired the relative accuracy of JOLs (ability to distinguish remembered from non-remembered items; Experiment 2). Acute exercise seems to signal likely remembering; this has implications for understanding the effects of exercise on metacognition, and for incorporating exercise into study routines.

Experience-based knowledge increases confidence in discriminating our memories

The present study investigated the accuracy of metacognitive judgments in source monitoring with self-report scales engaging either information- or experienced-based knowledge. We expected that the source monitoring abilities may be affected by the origins of meta-knowledge that underlie post-decision wagering (PDW) based on economic categorizations (experience-based scale) and confidence ratings (CR) using a conventional taxonomy of confidence (information-based scale). To examine this hypothesis, healthy participants (N = 50) performed an action memory task, in which simple actions were presented in order to be performed or imagined. In the second phase of the task, participants were required to assess source monitoring by distinguishing whether the presented action was performed or imagined. Then, the participants randomly assigned into the PDW or CR group rated their confidence in responses related to source monitoring performance. It was found that source monitoring ability is resistant to manipulation of the type of knowledge used in the scales. However, measures of metacognition indicated that accuracy of the experienced-based judgments of PDW was higher as compared to the CR scale while source monitoring. These findings suggest the origin of knowledge whose justification rests more on empirical observations generates more accurate knowledge than self-evident direct intuition with respect to discriminations of one’s own memories.

Predicting others' knowledge: Knowledge estimation as cue utilization.

Predicting what others know is vital to countless social and educational interactions. For example, the ability of teachers to accurately estimate what knowledge students have has been identified as a crucial component of effective teaching. I propose the knowledge estimation as cue-utilization framework, in which judges use a variety of available and salient metacognitive cues to estimate what others know. In three experiments, I tested three hypotheses of this framework: namely, that participants do not automatically ground estimates of others' knowledge in their own knowledge, that judgment conditions shift how participants weight different cues, and that participants differentially weight cues based upon their diagnosticity. Predictions of others' knowledge were dynamically generated by judges who weighed a variety of available and salient cues. Just as the accuracy of metacognitive monitoring of one's own learning depends upon the conditions under which judgments of self are elicited, the bases and accuracy of metacognitive judgments for others depends upon the conditions under which they are elicited.

When analogies harm: The effects of analogies on metacomprehension

The main goal of the present research was to test whether the presence of analogies would affect the relative accuracy of metacognitive judgments about learning from expository science texts, and whether any effect would depend on the type of cues that readers used as the basis for their judgments of comprehension. In a series of experiments, students read texts that either contained or did not contain analogies; were asked to judge how well they understood each text; took comprehension tests for each topic; and were asked to self-report the basis for their judgments. Relative metacomprehension accuracy was computed as the intra-individual correlation between judgments and test performance. Results showed that the presence of analogies can lead to poor relative metacomprehension accuracy for students who fail to use situation-model-based cues to judge their understanding of text.

Accuracy of metacognitive judgments as a moderator of learner control effectiveness in problem-solving tasks

A possible explanation for why students do not benefit from learner-controlled instruction is that they are not able to accurately monitor their own performance. The purpose of this study was to investigate whether and how the accuracy of metacognitive judgments made during training moderates the effect of learner control on performance when solving genetics tasks. Eighty-six undergraduate students solved self-selected genetics tasks using either a full learner control or a restricted learner control. Results indicated that learner control effectiveness was moderated by the absolute accuracy (i.e., absolute bias) of metacognitive judgments, and this accuracy was a better predictor of learning performance for full learner control than for restricted learner control. Furthermore, students’ prior knowledge predicted absolute accuracy of both ease-of-learning judgments (EOLs) and retrospective confidence judgments (RCJs) during training, with higher prior knowledge resulting in a better absolute accuracy. Overall, monitoring guided control, that is, EOLs predicted time-on-task and invested mental effort regardless of the degree of learner control, whereas RCJs predicted the total training time, but not the number of tasks selected during training. These results suggest that monitoring accuracy plays an important role in effective regulation of learning from problem-solving tasks, and provide further evidence that metacognitive judgments affect study time allocation in problem solving context.

Age-related similarities and differences in monitoring spatial cognition

ABSTRACTSpatial cognitive performance is impaired in later adulthood but it is unclear whether the metacognitive processes involved in monitoring spatial cognitive performance are also compromised. Inaccurate monitoring could affect whether people choose to engage in tasks that require spatial thinking and also the strategies they use in spatial domains such as navigation. The current experiment examined potential age differences in monitoring spatial cognitive performance in a variety of spatial domains including visual–spatial working memory, spatial orientation, spatial visualization, navigation, and place learning. Younger and older adults completed a 2D mental rotation test, 3D mental rotation test, paper folding test, spatial memory span test, two virtual navigation tasks, and a cognitive mapping test. Participants also made metacognitive judgments of performance (confidence judgments, judgments of learning, or navigation time estimates) on each trial for all spatial tasks. Preference for allocentric or egocentric navigation strategies was also measured. Overall, performance was poorer and confidence in performance was lower for older adults than younger adults. In most spatial domains, the absolute and relative accuracy of metacognitive judgments was equivalent for both age groups. However, age differences in monitoring accuracy (specifically relative accuracy) emerged in spatial tasks involving navigation. Confidence in navigating for a target location also mediated age differences in allocentric navigation strategy use. These findings suggest that with the possible exception of navigation monitoring, spatial cognition may be spared from age-related decline even though spatial cognition itself is impaired in older age.

Metacognitive judgments and disfluency – Does disfluency lead to more accurate judgments, better control, and better performance?

Theories of metacognition assume that better monitoring leads to better control and performance. Furthermore, monitoring accuracy is often low because students are overconfident (absolute accuracy) and unable to discriminate comprehension of different text-passages (relative accuracy). Fluency seems to be a cue for metacognitive judgments, and therefore, reducing fluency should lead to less automatic processing, lower judgments, and better absolute and relative accuracy. Because the accuracy of metacognitive judgments is the basis of the control of learning, disfluency should lead to more appropriate control and thus to better performance. To test these assumptions, students (N = 83) learned either with disfluent or with fluent text-passages. The results show that disfluency led to better absolute and relative accuracy but not for all types of judgments. Moreover, students hardly implemented monitoring in control, resulting in lack of improved performance. Further research is required to investigate why students did not base control on monitoring.

Calibrating Genre: Metacognitive Judgments and Rhetorical Effectiveness in Academic Writing by L2 Graduate Students

Several strands of applied linguistic research have emphasized the importance of genre awareness for academic writing students. Although metacognitive behaviors have been linked to L2 writing proficiency and performance, there is still the need for an account of how and why different metacognitive behaviors can help L2 academic writers to apply genre knowledge in authentic situations. Adopting an interdisciplinary approach, this study borrows the framework of calibration from educational psychology to highlight the relationship between the accuracy of graduate students’ metacognitive judgments and the quality of their texts. Within an authentic setting, the nature of metacognitive judgments is calibrated against the assessment of rhetorical effectiveness by teacher raters using genre analysis criteria. Findings show that individual differences in rhetorical effectiveness can be better understood when accuracy of metacognitive judgments is considered along two qualitative dimensions: depth and alignment. Differential achievement relates to the ability to apply genre knowledge to the text, and misalignments in task perceptions and criteria. Implications for genre pedagogy and further research are discussed.

Accuracy of teachers’ tracking decisions: short- and long-term effects of accountability

Bias in teachers’ judgment formation and decision making has long been acknowledged. More specifically, studies have repeatedly demonstrated discrepancies between teacher ratings of minority and majority students with similar academic profiles. Studies have also demonstrated that increasing accountability reduced bias. Little is known, however, about the effect of accountability and bias on the accuracy of decisions. This study investigated the short- and long-term effect of accountability priming on the accuracy of transition decisions. It considered both the extent to which teacher decision accuracy differed for minority and majority students with similar academic profiles (accuracy bias) and differences in levels of confidence for accurate versus erroneous decisions (metacognitive judgment bias). In a longitudinal experimental design, we presented 38 primary school teachers with 9 student vignettes at 3 points in time (baseline, post priming, and 6-month follow-up), varying students’ ethnic background, and asked them to make a school tracking decision for each student. We measured decision accuracy as well as teachers’ level of confidence for each decision. Accuracy and confidence levels were combined to provide two indices of metacognitive judgment accuracy. Results confirmed the hypothesis that accuracy of decisions would improve as a result of increased level of accountability. More specifically, we found that teachers made more accurate decisions after priming, whereby ethnic background differences disappeared. In addition, teachers’ metacognitions varied, whereby after priming decision accuracy was better matched with teachers’ confidence levels. Although accuracy levels were still higher at follow-up than at pre-test, the ethnic bias recurred. This study shows that increased levels of accountability are associated with not only increased decision accuracy but also reduced metacognitive judgment bias, especially in regard to minority students. It also demonstrates accountability may be an effective way of reducing systematic errors in decision making. Findings are discussed in terms of theory and current changes in educational practice.

Метакогнитивный мониторинг знания в условиях оценки учебных достижений

We present an empirical study, which tested the hypothesis about the impact of the situation of educational achievements assessment on metacognitive monitoring of students’ subject knowledge test performance. On a sample of 47 people (21 girls, 26 boys, mean age 15.5 years) we revealed that the characteristics of metacognitive monitoring (self-decision and the accuracy of metacognitive judgments) differ in a situation of educational achievements assessment and in its absence. We found that students with high and low achievement motivation show different metacognitive monitoring of the situation in the presence or absence of the estimation of educational achievements (p < 0,05). In conclusion, we discuss the use of the data and findings in the general psychological and psycho-pedagogical studies of metacognition, and in teaching practice.

Metacognitive monitoring during criterion learning: When and why are judgments accurate?

Research on metacognitive judgment accuracy during retrieval practice has increased in recent years. However, prior work had not systematically evaluated item-level judgment accuracy and the underlying bases of judgment accuracy in a criterion-learning paradigm (in which items are practiced until correctly recalled during encoding). Understanding these relationships during criterion learning has important theoretical implications for self-regulated learning frameworks, and also has applied implications for student learning: If the factors that influence metacognitive judgments are not predictive of subsequent test performance, students may make poor decisions during self-regulated learning. In the present experiments, participants engaged in test-restudy practice until items were recalled correctly. Once a given item reached criterion, participants made an immediate or delayed judgment of learning (JOL) for the item. A final cued-recall test occurred 30 min later. We examined judgment accuracy (the relationship between JOLs and test performance) and the underlying bases of judgment accuracy by evaluating cue utilization (the relationship between cues and JOLs) and cue diagnosticity (the relationship between cues and test performance). Immediate JOLs were only modestly related to subsequent test performance, and further analyses revealed that the cues related to JOLs were only weakly predictive of test accuracy. However, delaying JOLs improved both the accuracy of the JOLs and the diagnosticity of the cues that influenced judgments.